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A Report on the Implementation of the Blooming Biology Tool: Aligning 
Course Learning Outcomes with Assessments and Promoting 
Consistency in a Large Multi-Section First-Year Biology Course 


Abstract 

The objectives of this study were to investigate the alignment of exam questions with course learning 
outcomes in a first year biology majors course, to examine gaps and overlaps in assessment of content amongst 
the sections of the course, and to use this information to provide feedback to the teaching team to further 
improve the course. Our ultimate goal was to provide students with learning outcomes that would clearly 
indicate the content and the level at which they would be expected to learn the content for this course, 
regardless of the section in which they were registered. We took an evidence-based approach to course 
evaluation and employed the Blooming Biology Tool to compare the learning outcomes and the exam 
questions of the course, investigating whether the cognitive skill level of each learning outcome as written 
matched the level at which it was assessed. We identified misalignments and recommended revising the 
learning outcomes to better reflect the intended level of learning for the course. We also investigated student 
performance on exam questions of different cognitive levels and found that students scored statistically 
significantly higher (p < .05) on questions in which learning outcomes were tested at the stated cognitive skill 
level compared to at a higher level. 

Les objectifs de cette etude etaient (1) d’examiner la correspondance entre les questions d’examen et les 
resultats en matiere d’apprentissage pour un cours de premiere annee dune majeure en biologie, (2) d’etudier 
les ecarts et les chevauchements en matiere d evaluation du contenu des sections du cours et (3) d’utiliser ces 
informations pour fournir de la retroaction a l’equipe des enseignants afin d’ameliorer le cours. Notre but 
ultime etait de faire en sorte que les resultats de l’apprentissage des etudiants indiquent clairement le contenu 
a apprendre et le niveau cognitif qu’ils devraient avoir atteint, peu importe la section a laquelle ils s etaient 
inscrits. Nous avons utilise une approche basee sur les donnees probantes pour evaluer le cours ainsi que 
l’outil de taxonomie de Bloom applique a biologie pour comparer les resultats d’apprentissage et les questions 
d’examen du cours. Nous souhaitions ainsi verifier si le niveau de competences cognitives tel qu’il est ecrit 
pour chaque resultat d’apprentissage correspondait au niveau auquel il etait evalue. Nous avons decouvert des 
correspondances inadequates et avons recommande de reviser les resultats d’apprentissage pour mieux refleter 
le niveau d’apprentissage souhaite dans le cours. Nous avons egalement etudie la performance des etudiants 
aux questions d’examens en fonction de differents niveaux cognitifs et avons decouvert que les resultats des 
etudiants etaient significativement plus eleves sur le plan statistique (p < 0,05) pour les questions ou les 
resultats d’apprentissage etaient verifies au niveau des competences cognitives declare, plutot qua un niveau 
plus eleve. 
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Learning outcomes describe what learners will be able to do on completion of a particular 
learning experience. Having well-articulated learning outcomes in a course eliminates student 
uncertainty and anxiety about what they are expected to know, helps students to prepare for 
assessment, and allows faculty to design assessment questions that are in alignment with the 
intended learning for the course. For course learning outcomes to be a useful tool for guiding 
student learning, these learning outcomes must list not only the topics that students will be 
responsible for learning in the course but also the cognitive level at which the students will be 
assessed for each of these topics. 

One tool used to rank the cognitive level of learning outcomes is the cognitive domain of 
Bloom’s Taxonomy, which ranks thinking and knowledge into six categories. The Knowledge (I) 
and Comprehension (II) levels involve mainly lower order cognitive skills such as recognizing, 
recalling, and explaining memorized information (Bloom, 1956). Analysis (IV), Synthesis (V), 
and Evaluation (VI) levels require the use of higher order cognitive skills such as problem 
solving and critical thinking (Bloom, 1956; see also Anderson & Krathwohl, 2001). Application 
(III) is often considered to be a transition between lower and higher order cognitive skills 
(Crowe, Dirks, & Wenderoth, 2008). Most biology questions at the Application level require the 
use of critical thinking and problem solving skills. For example, a typical Application biology 
question might provide students with an unfamiliar scenario and ask them to predict the outcome 
of altering one or more of the relevant variables. To answer this question, students cannot simply 
rely on memorization and understanding of previously learned concepts (lower order cognitive 
skills), but must be able to use what they already know to solve this novel problem. At this point, 
the use of lower order cognitive skills such as memorizating and recalling in the problem solving 
process is minimal, and higher order cognitive skills are the dominant skills used. To that end, 
for the purpose of our study, we considered the Application (III) level in the higher order 
cognitive skills category, which is consistent with other disciplines (e.g., Fuller, 1997). 

The evidence-based approach to teaching and learning in biology has introduced tools to 
the teaching community to effectively evaluate their course designs. One such tool, recently 
published by Crowe et al. (2008), is the Blooming Biology Tool. The Blooming Biology Tool is 
an evaluation instrument derived from Bloom’s Taxonomy and is applicable to any biology- 
related topic. Using this tool, assessment questions can be categorized into one of the six 
cognitive skill levels described above. Applications of the Blooming Biology Tool include 
aligning course assessment tools with teaching activities and learning outcomes, as well as 
helping students to enhance their study skills and metacognition (Crowe et al., 2008). 

The importance of aligning course activities with assessments and learning outcomes has 
been well articulated in the literature (Bissell & Lemons, 2006; Crowe, et al., 2008; Ebert-May, 
Batzli, & Lim, 2003; Fink, 2003; Sundberg, 2002; Tanner & Allen, 2004; Wiggins & McTighe, 
1998). A detailed report by Bateman, Taylor, Janik, and Logan (2007) documented a process that 
integrated the measurement of the assessment of outcomes with instructional objectives and the 
classroom assessments that are designed to measure the attainment of these objectives. A recent 
publication by Zheng, Lawhorn, Lumley, and Freeman (2008) examined Bloom’s level of exam 
questions in first year biology courses, comparing the level of thinking required in these 
questions with questions from Advanced Placement (AP) exams, Graduate Record Exams 
(GREs), Medical College Admission Tests (MCATs), and first year medical school courses. The 
alignment of course material with learning outcomes and assessments becomes particularly 
complicated when the course has multiple sections. Maintaining consistency of content and 
assessment across multiple sections of the course requires that the instructors of the course 
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communicate closely with each other and that there is a team-teaching approach, ideally towards 
a common exam. An example of a significant transformation of a multisection physics course is 
illustrated elsewhere (Gladding, 2007). 

The Biology Program at the University of British Columbia is delivered through the 
Botany and Zoology Departments. It includes introductory biology courses in first and second 
year, core upper-level biology courses in genetics, evolution, ecology, cell biology, physiology, 
and biometrics, and elective courses in many subject areas of biology. Based on our own 
experience, in the last decade students have become increasingly adept at finding information 
electronically. It was reported that by far the most frequently cited school-related use of the web 
was to do research and/or get information (mentioned by 89% of the respondents) in a study 
conducted by Metzger, Flanagin, and Zwarun (2003). As a result of the type of examinations 
requiring students to respond to lower levels of cognitive thinking, students have become 
exceptionally good at regurgitating information on exams. However, they are increasingly 
lacking in written and analytical skills. Therefore, in first year courses, we have increased our 
emphasis on hypotheses testing and data analysis in lectures. To reflect this tendency, our exams 
have become more focused on application of infonnation and less focused on memorization of 
facts. These exams are often more concept based than the exams that many students may have 
experienced in high school. In addition, most instructors now make use of content-based 
questions in web-based preclass assignments and in-class clicker questions. Thus, to help 
students prepare for concept-based exams, it is necessary to set clear learning outcomes for the 
students that align with the exam questions. 

One good example of the above is our Biology 121 course. This course is a first-year, 
multisection course, which introduces ecology, evolution, and genetics; it is a required course in 
the UBC Biology Program and is also required in programs in other faculties (e.g., Forestry). 
This course consists of 10 lecture sections, each section taught by a different faculty member, 
and is offered to about 2,000 students annually. Recently, within the context of curriculum 
revisions in the Biology Program at UBC, a team of faculty members teaching Biology 121 
developed learning outcomes for the course. (These learning outcomes are provided in the 
Appendix.) The first objective for our study was to apply the Blooming Biology Tool to evaluate 
the alignment of these course outcomes with the exam questions in the different sections of this 
course. Our second objective was to use the results of this evaluation to revise the course 
learning outcomes to better reflect the intended learning as well as to promote consistency across 
all sections of this course. Our third objective was to analyze students’ final exams to investigate 
whether the cognitive skill level of an assessment question affected student perfonnance on that 
question and whether assessing a given learning outcome at a higher cognitive skill level than 
stated would impact student performance. We tested the hypothesis that students would earn 
higher marks on questions that were in agreement with the cognitive skill level of a learning 
outcome than on questions that were at a higher cognitive skill level than stated in that learning 
outcome. The results of this study have been valuable in helping our faculty members make 
informed curricular decisions with respect to the course and will potentially alleviate the 
frustration of both students and faculty due to the misalignment of learning outcomes and 
assessment. The ultimate goal of this contribution is to share our experiences with other 
educators, demonstrating the feasibility of using this process to align a course or program 
curriculum with the intended student learning experience. 
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Methods 


Data Collection 

In this study, we used Biology 121 course learning outcomes, midterm and final exams 
provided by the instructors of four different sections of the course (Sections A, B, C, and D, 
arbitrarily labeled to preserve anonymity, offered from January to April of 2008), and student 
grades that corresponded to each question on the final exam from Section B. We followed an 
ethics protocol on a companion study to measure student attitudes and beliefs approved by 
Behavioural Research Ethics Board as required by UBC, and we used only those students who 
consented for their grades to be used in this study. The flowchart of the process is shown in 
Figure 1. Original learning outcomes and the exam questions were rated for their cognitive skill 
level. The exam questions were then mapped onto the learning outcomes in order to align them. 
The course learning outcomes were then revised to better reflect the intended cognitive skill 
levels to be measured in the course. Exams were open-book in Sections A and C, and were 
closed-book in Sections B and D. 



Figure 1. The summary flowchart of the course evaluation process employed in the study. 

We recorded the number of marks allotted to each question and the total marks for each 
exam. Because the mark total for each exam did not add up to 100, we totaled the number of 
marks available for all exams and calculated the relative weight of each exam question to 
calculate the percentage of the total available exam marks for each question in that section. For 
example, in Section B, there were a total of 204 available exam marks (midterm 1, 50 marks; 
midterm 2, 36 marks; and final exam, 118 marks) that a student could earn. Therefore a “one 
mark” exam question was worth approximately 0.5% of the total available exam marks for this 
section. 

We categorized exam questions into three types: multiple choice, short answer, or 
paragraph. For our purposes, we defined multiple choice type questions as questions for which 
students had to choose one or more correct answers from a list of several possible answers. We 
defined short-answer type questions as questions for which students were required to write a 
response ranging in length from one word to several sentences, draw a graph or diagram, or 
produce a concept map. We defined paragraph-type questions as questions for which students 
were required to write a response longer than several sentences and for which some portion of 
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their mark for that question would be based on their writing style. We then further grouped short 
answer and paragraph answer questions into a single group called written answer questions and 
compared them with multiple choice questions as explained in the Comparison of Multiple 
Choice and Written Answer Questions Section below. 

Based on the relative weight of each exam question, we calculated the percentage of all 
available exam marks tested using each question type in each section. We compared these 
percentages amongst the four sections to determine if students were tested to the same degree 
with the same question types in different sections of the course. 

Blooming Exam Questions 

For each section, we rated each exam question based on the six cognitive skill levels 
originally described by Bloom (1956) using the Blooming Biology Tool as a rubric (Crowe et ah, 
2008). This process of rating is referred to as blooming. This tool provides biology-specific 
criteria for blooming questions (multiple choice and written answer) at each cognitive skill level 
and includes specific examples of commonly used questions. This tool allowed us to bloom any 
biology question independently and consistently by simply detennining where it best fit in the 
criteria. The example questions provided in the tool further served us as a guide in this blooming 
process. 

We considered all of the cognitive skills a student would need to employ to correctly and 
completely answer that question, relying on the marking key provided by the instructors, and 
assigned the highest cognitive skill level required to that question, consistent with the procedure 
described by Crowe et al. (2008). For example, consider the question “Why can’t natural 
selection act on rocks?” This question requires that students know the definition of natural 
selection (Knowledge -1), be able to explain the process of natural selection in their own words 
(Comprehension - II), and apply this general information to the specific situation involving rocks 
(Application - III). We therefore categorized this as an Application (III) question. Additionally, 
we also considered how the material that was being tested in a specific question was presented in 
class by the instructor (Crowe et al., 2008). For example, a question requiring students to design 
an experiment would normally be categorized as a Synthesis (V) question (Crowe et al., 2008). 
However, if the exam question asked students to design an experiment, but the instructor had 
already presented the design for that experiment in class, then students need only recall that 
information-a Knowledge (I) question (Crowe et al., 2008). 

Based on the Blooming Biology Tool, we classified questions in the first two levels of 
Bloom’s Taxonomy, Knowledge (I) and Comprehension (II), as questions requiring only lower 
order cognitive skills, and Analysis (IV), Synthesis (V), and Evaluation (VI) as questions 
requiring higher order cognitive skills (Crowe et al., 2008). We also considered Application (III) 
questions as requiring higher order cognitive skills (as explained in the Introduction section) 
since most biology questions at this level require students not only to apply information that they 
have learned but also to use their critical thinking skills. For example, Exam Question B in Table 
1 requires students to predict the likelihood of a captive breeding program to restore the 
population of an endangered species. To do this, they must consider how various factors such as 
inbreeding and genetic drift will impact the allele frequencies in this population and whether this 
restricted genetic diversity will prevent this species from surviving. They must first identify 
which factors are relevant to this specific situation and understand how each of these factors 
would affect gene frequencies in a population, which would require only recall and basic 
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understanding of these concepts. However, they must then draw conclusions through critical 
thinking, weighing the relative impact of each of these factors to make a prediction about the 
outcome of this scenario. 


Table 1 

Examples of Recommended Wording Modifications to Better Align the Cognitive Skill Level of 
Learning Outcomes and Exam Questions, and of Streamlining Through the Merging of Several 
Rel ated Learning Outcomes _ 


Sample Description 


Cognitive 
skill level 


Learning Outcome 1 Describe how biologists study the history of the diversity of life on earth. II 


Sample Exam 
Question A 


Recommended 
Learning Outcome 1 


Having reached the Planet Zogor in a distant galaxy, humans disagree as to 
the origin of life on the planet. Some say it was planted on many occasions 
from distant galaxies. Others think it arose and diversified on the planet. 
What biological data would you collect to prove or disprove either 
hypothesis? (Use the earth's biodiversity as a model.) 

Design tests to investigate the evolution of unknown life forms, based on 
practices currently or historically used by biologists to study the history of y 
the diversity of life on earth. 


Learning Outcome 2 


Sample Exam 
Question B 


Identify the sources of variation within populations. a. II 

a. describe the types of selection and detennine how they affect variation 

in populations. b. II 

b. Describe four processes that contribute to nondirectional changes in 

allele frequencies: genetic drift, gene flow, mutation, and inbreeding. c. I 

c. Identify how each of these processes contributes to changing allele 
frequencies in a population. 

Lonesome George (the last living member of a subspecies of Galapagos 
tortoise) may have a potential mate; a subspecies hybrid has recently been 
found. She is 1/2 of George's subspecies. If they do breed, do you expect that 
the subspecies will escape an extinction vortex? Why? (Diagram and explain 
an extinction vortex as part of your answer.) 


Recommended 
Learning Outcome 2 


Predict how sources of variation in populations (including different types of 
selection, genetic drift, gene flow, mutation, and inbreeding) will contribute III 
to changing allele frequencies in a population. 
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Table 1 (continued) 


Sample Description 


Cognitive 
skill level 


Describe how abiotic and biotic factors affect population and community 

T „ ^ _ structure and evaluate the importance of these factors in specific 

Learning Outcome 3 . . VI 

communities. 


A study on the effects of sulfide on intertidal ecosystems scored intertidal 
organisms on the upper and lower surfaces of rocks on beaches with and 
without sulfide. The results of this study are shown in the following graph. 
Which of the following statements are supported by this data? 

• Control beaches have more rocks than sulfide 
beaches. 

• Control beaches have more species than sulfide 
beaches. 

• Rocks from control beaches have more species on 
their lower surfaces than on their upper surfaces. 

• Rocks from control beaches have more biomass 
on their lower surfaces than on their upper surfaces. 

• Rocks from sulfide beaches have the same species 
on their upper and lower surfaces. 

Figure A: Number of distinct species on upper and 
lower surface of rocks in control and sulfide 
beaches. Data points are means of six replicate 
rocks, bars are 95% confidence intervals (if these 
do not overlap the means are significantly 
different). 



Recommended Analyze how biotic and abiotic factors affect population and community 

Learning Outcome 3 structure, and the importance of these factors in specific communities. 


The Blooming Biology Tool has already been used by others to rank hundreds of exam 
questions in the life sciences with great consistency (Zheng et al., 2008). To ensure consistency 
in our rating of exam questions, one rater bloomed all of the exams in this study, using the 
Blooming Biology Tool as a rubric (Crowe et al., 2008). To assess the intrarater reliability of this 
primary rater, we had this rater bloom one representative final exam (Section A) again, 2 months 
after the original rating. This final exam contained a total of 19 multiple choice and 9 short 
answer questions. This rater demonstrated a 90% agreement between the two ratings. 

To confirm the reliability of these ratings, a second rater bloomed the same representative 
final exam (Section A) independent from the primary rater within the same time frame, also 
using the Blooming Biology Tool as a rubric (Crowe et al., 2008). These two raters then 
compared their ratings and agreed for 83% of the exam questions. For all questions where their 
ratings differed, these two raters discussed the ratings and came to a consensus. The primary 
rater then reviewed all of the original ratings, making adjustments as necessary to reflect the 
consensus reached through discussion with the second rater. Both raters were teaching faculty 
members in the Biology Program at UBC and already had extensive experience working together 
in writing and grading exam questions. They trained for this rating process by reviewing the 
Blooming Biology Tool, which provided specific examples from the biology discipline. They 
then practiced blooming sample exam questions using this tool as a rubric, comparing and 
discussing their ratings afterwards. 

For each section, we determined the percentage of available exam marks that students 
could earn at each cognitive skill level. We combined all exam questions (midterms 1 and 2, and 
final) to produce the overall available exam mark distribution across each cognitive skill level for 
each section. 
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Measuring Assessment of Course Learning Outcomes 

For each section, we mapped each exam question to the appropriate course learning 
outcome(s). We then calculated the percentage of all available exam marks allotted to each 
course learning outcome for each section. If a question tested more than one learning outcome, 
we divided the marks for that question equally amongst all learning outcomes tested. We 
compared these percentages amongst the four sections to detennine whether students were 
examined on the same learning outcomes in different sections of the course and whether these 
learning outcomes were given approximately the same weight on exams in different sections of 
this course. These results allowed us to identify the gaps and overlaps in the examination of 
course content amongst the different sections of this course. 

We then determined at which cognitive skill level each learning outcome was assessed in 
each section. Often individual learning outcomes were assessed in more than one exam question 
per section, and often not all questions assessing the same learning outcome were written at the 
same cognitive skill level. In this situation, we determined the cognitive skill level at which that 
learning outcome was most frequently assessed based on the mark value of each question used to 
assess that learning outcome. For each section, we then compared the cognitive skill level at 
which each course learning outcome was most frequently assessed to the level at which that 
same learning outcome was stated in the list of learning outcomes provided to the students that 
tenn. 

Measuring Student Performance on Exam Questions 

We used the average mark earned by students on each exam question to compare student 
performance on final exam questions of different cognitive skill levels for the 101 students in 
Section B who consented to let us use their grades in this study. We also compared the average 
marks earned by students on exam questions that assessed a given learning outcome at its stated 
cognitive skill level to the marks earned on exam questions that assessed that same learning 
outcome at a higher level. We used a one-sided Student’s paired t test to compare the significance 
of the difference in exam marks between questions examining a given learning outcome at its 
stated cognitive skill level and questions examining that same learning outcome at a higher level 
and accepted a p value of < .05 as the level of statistical significance. Furthermore, we compared 
the performance on these questions for the top 25% of students in this section (the 25 students 
who earned the highest total mark on the final exam out of the 101 students who consented to let 
us use their grades in this study) and for the bottom 25% (the 25 students who earned the lowest 
total mark on the final exam out of the 101 students who consented to let us use their grades in 
this study). 
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Results and Discussion 

Comparison of Cognitive Skill Levels of Midterm and Final Exams 

Midterm exams. Figure 2 shows the available exam mark distribution across each 
cognitive skill level for midterm 1, midterm 2, and the final exam for Section B of the course. 
The percentage of available marks requiring the use of higher order cognitive skills increased 
from 28% to 56% between the first and the second midterms (see Figure 2). However, while the 
number of Application questions increased between the first (26%) and second (56%) midterm, 
the number of Analysis questions decreased (2% on the first midterm and 0% on the second 
midterm). In this section, there were no Synthesis ox Evaluation questions on the midterms. 



c I II III IV v VI 

Bloom's Taxonomy Level 

Figure 2. Available exam mark distribution across each cognitive skill level (i.e., Bloom’s 
Taxonomy Level): Progression from midterm to final exam for a representative section of 
Biology 121 (Section B, Jan-Apr 2008). A: Midtenn 1. B: Midterm 2. C: Final Exam. The six 
levels of the cognitive domain of Bloom’s Taxonomy are: Knowledge (I), Comprehension (II), 
Application (III), Analysis (IV), Synthesis (V), and Evaluation (VI). 
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Sections A and D also had two midterms while Section C had only one. Unlike in Section 
B, in Section D the percentage of available marks requiring the use of higher order cognitive 
skills remained unchanged from the first midterm (85%) to the second midterm (85%). Section A 
showed a decrease in the percentage of available marks requiring the use of higher order 
cognitive skills from the first midterm (67%) to the second midterm (54%); however, the 
percentage of Analysis questions increased from the first midterm (20%) to the second midterm 
(30%) while the percentage of Knowledge questions decreased from the first midterm (20%) to 
the second midterm (0%). All sections studied had Comprehension, Application, and Analysis 
midterm questions. However, only Sections C and D had Synthesis midterm questions, while 
only Sections A and B had Knowledge midterm questions. 

Final Exams. The percentage of all available marks requiring higher order cognitive 
skills on the Section B final exam was 75%, which was substantially higher than for either 
midterm (28% and 56%) in this section. For this final exam, the percentage of available marks 
for Analysis was 39%, 36% for Application, 16% for Comprehension, and only 9% for 
Knowledge. The final exam had no Synthesis or Evaluation questions. 

In Section A, the percentage of all available marks requiring higher order cognitive skills 
on the final exam (65%) was similar to that on the first midterm (67%) and higher than that on 
the second midterm (54%). In Section C, the percentage of all available marks requiring higher 
order cognitive skills was similar on the midterm (80%) and the final exam (77%), while in 
Section D, the percentage of all available marks requiring higher order cognitive skills decreased 
from the midterms (both 85%) to the final exam (74%). Students have over twice as much time 
available to write final exams as to write midterms. Because the additional time for the final 
exam gives instructors the opportunity to include questions that are more complex and require 
more in-depth analysis than do midterm questions, we expected that all sections studied would 
have a higher percentage of available marks requiring higher order cognitive skills on the final 
exam than on the midterms. However, for Sections C and D, the percentage of available marks 
requiring higher order cognitive skills on the final exam was equal to or less than that of the 
midterms. We anticipated that this would be due to the already high percentage of available 
marks requiring higher order cognitive skills on the midterms in Sections C and D. Additionally, 
the instructors in all sections of the course tried to always include a few Comprehension 
questions on the final exam to help students who were struggling with the course material, as 
these students generally performed better on Comprehension questions than they did on 
questions requiring higher order cognitive skills. Because of the difficulties in having Evaluation 
questions in the time-constrained setting of an exam, even a final exam, Synthesis was the 
highest cognitive skill level examined in this course. Therefore, for Sections C and D, in which 
instructors were already using Synthesis questions on the midterms, and were also minimizing 
questions requiring only lower order cognitive skills on the midterms, there was little room to 
increase the number of questions requiring higher order cognitive skills on the final exam as 
compared to the midterm exams. 

Comparison of Cognitive Skill Level Distribution of Exam Marks amongst Four Sections 

We combined the midterm and final exam data for each section in order to evaluate the 
overall exam mark distribution across each cognitive skill level for each section, then compared 
amongst the four sections to detennine if students were examined at approximately the same 
cognitive skill level in these four sections of the course (Figure 3). Only Sections A (4%) and B 
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(15%) had Knowledge exam questions. Section A had the highest percentage of Comprehension 
exam marks (34%), followed by Section B (25%), Section C (21%), and Section D (20%). 
Section D had the highest percentage of Application marks (55%), followed by Section B (37%), 
Section A (31%), and Section C (24%). Section C had the highest percentage of Analysis marks 
(40%), followed by Section A (23%), Section B (23%), and Section D (19%). Only Sections A, 
C, and D had Synthesis questions, and no sections had Evaluation questions. Section C had the 
highest percentage of Synthesis marks (16%), followed by Section A (8%) and Section D (7%). 

In all four sections studied, the percentage of available exam marks requiring higher 
order cognitive skills was calculated to be between approximately 60% (Sections A and B) and 
80% (Sections C and D). Although Sections C and D had a nearly identical percentage of exam 
marks requiring higher order cognitive skills, they differed in that only 30% of these exam marks 
in Section C were at the transitional Application level, compared to 69% in Section D. Similarly, 
Sections A and B differed in that only 50% of exam marks requiring higher order cognitive skills 
in Section A were at the transitional Application level, compared to 62% in Section B. Our 
findings revealed that the cognitive skill level of questions in this first year biology course at 
UBC is slightly higher than that of the first year biology courses in the three U.S. universities 
sampled by Zheng et al. (2008) in their study of undergraduate biology and first year medical 
school biology courses as well as of AP Biology, GRE, and MCAT exam questions. In our case, 
the Analysis level questions were approximately 30% on average as opposed to approximately 
10% for the first year biology courses sampled in these three U.S. universities, while Knowledge 
level questions were ~8% on average in our case as opposed to 20% for them. The weight of the 
Application and Comprehension level questions was about the same in both cases. 
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Figure 3. Comparison of four sections of the course (Jan-Apr 2008) with respect to available 
exam mark distribution across each cognitive skill level (i.e., Bloom’s Taxonomy Level). LOGS: 
lower order cognitive skills, HOCS: higher order cognitive skills. The six levels of the cognitive 
domain of Bloom’s Taxonomy are: Knowledge (I), Comprehension (II), Application (III), 
Analysis (IV), Synthesis (V), and Evaluation (VI). 
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The differences in exam mark distribution across each cognitive skill level amongst the 
four sections suggest that students in different sections of the course were not examined at an 
equivalent cognitive skill level. To alleviate the differences amongst sections, last year course 
instructors wrote several questions (equivalent to approximately 40% of the final exam marks) 
that would be common to the final exams of all sections of this course and implemented this 
change from January to April 2009. Our results confirmed the need for these common exam 
questions to promote consistency in assessment amongst the different sections of this course. 

The format of these exams being either open-book (Sections A and C) or closed-book 
(Sections B and D) did not appear to affect the distribution of marks across cognitive skill levels. 
For example, although Section D had closed-book and Section C had open-book exams, the 
percentage of questions requiring higher order cognitive skills was comparable, and neither 
section had any Knowledge level exam questions. 

Comparison of Multiple Choice and Written Answer Questions 

The exams of the four sections studied varied not only in the distribution of exam marks 
across each cognitive skill level but also in the use of different question types. Section A had the 
largest percentage of exam marks for multiple choice (37%), closely followed by Section C 
(35%), and Section B (22%), while Section D exams had no multiple choice questions. Only two 
sections had paragraph exam questions: Section B (5%) and Section C (21%). Section D had the 
highest percentage of exam marks for short answer questions (100%), followed by Sections B 
(73%), A (63%), and C (44%). Because paragraph answers require students to demonstrate more 
developed writing skills than do short answer questions, while multiple choice questions require 
no writing skills at all, these differences in question type amongst the sections place a different 
emphasis on the assessment of student writing skills. To address this discrepancy, we suggested 
that the course teaching team consider writing a list of course learning outcomes addressing 
skills such as writing in addition to the existing list of conceptual learning outcomes. 

For each section, we also determined the exam mark distribution across each cognitive 
skill level for multiple choice questions compared to other question types. We combined data for 
short answer and paragraph answer questions into a single category called written answer 
questions. For the three sections that had both multiple choice and written answer exam 
questions (Sections A, B, and C), 12% of all written answer marks were for Synthesis questions, 
while there were no multiple choice questions at this level (Figure 4). Only 14% of all multiple 
choice marks were for Analysis questions, compared to 36% of all written answer marks. More 
multiple choice marks were available (39%) at the Application level than were written answer 
marks (26%). Similarly, at the Comprehension and Knowledge levels, more multiple choice 
marks were available (36% and 11% respectively) than were written answer marks (22% and 4% 
respectively). 
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Figure 4. Available mark distribution across each cognitive skill level (i.e., Bloom’s Taxonomy 
Level): A Comparison of multiple choice and written answer questions (averaged data from 
Sections A, B, and C which had both types of questions in their exams, Jan-Apr 2008). Error 
bars indicate the standard deviation. The six levels of the cognitive domain of Bloom’s 
Taxonomy are: Knowledge (I), Comprehension (II), Application (III), Analysis (IV), Synthesis 
(V), and Evaluation (VI). 


Over half (53%) of the multiple choice marks were for questions requiring higher order 
cognitive skills, clearly demonstrating that it is possible to assess problem solving skills using 
multiple choice questions, as has previously been demonstrated for both biology (Zheng et al., 
2008) and for physics (Scott, Stelzer, & Gladding, 2006). However, the percentage of written 
answer marks (74%) requiring higher order cognitive skills was still much higher than that for 
multiple choice questions in our case. Zheng et al. (2008) showed that tests with carefully crafted 
multiple choice questions like GREs and MCATs had as high or higher cognitive skill level of 
questions than that of first year biology courses. Our results are in alignment with those results. 
We propose two solutions to alleviate potential student performance discrepancies among 
different sections of the course: One is to increase the proportion of common exam questions as a 
short-term solution, and two is to create validated multiple choice questions as a long-tenn 
solution. We have recently initiated a research project to address the latter in our institution. 


Cognitive Skill Level of Learning Outcomes 

During the January-April 2008 term, Biology 121 had 57 learning outcomes addressing 
the topics of genetics, evolution, and ecology. Genetics had 20 learning outcomes, of which 12 
were core and 8 were optional, while evolution had 22 learning outcomes, of which 8 were core 
and 14 were optional. Ecology had 13 learning outcomes divided into three subtopics: Patterns of 
Biodiversity, Population/Community Ecology, and Ecosystem Ecology. At the beginning of the 
tenn, instructors agreed to cover at least two of the three ecology subtopics, with the 
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understanding that they need not cover all the learning outcomes in each subtopic so long as 
students were given a broad perspective on ecological issues. Because of this arrangement, none 
of the 13 learning outcomes in ecology could be fully considered core outcomes, nor could they 
be considered fully optional outcomes. For the purposes of our study, we defined all ecology 
learning outcomes as core outcomes. Under this definition, Biology 121 then had 36 core 
learning outcomes and 22 optional learning outcomes. 

Of the 57 course learning outcomes, over 50% were written at the Comprehension level 
(Figure 5). As written, only 39% of the learning outcomes required higher order cognitive skills. 
However, in all sections studied, at least 60% of available exam marks required higher order 
cognitive skills. The number of course learning outcomes that were examined by questions 
requiring higher order cognitive skills was therefore much larger than what students would have 
expected, based on the wording of these learning outcomes. This difference could potentially 
confuse students. 
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Figure 5. Distribution of course learning outcomes across each cognitive skill level (i.e.,Bloom’s 
Taxonomy Level): A comparison of original course learning outcomes (A) from Jan-Apr 2008 to 
recommended learning outcomes (B). The six levels of the cognitive domain of Bloom’s 
Taxonomy are: Knowledge (I), Comprehension (II), Application (III), Analysis (IV), Synthesis 
(V), and Evaluation (VI). 

Assessment of Learning Outcomes 

No section of the course assessed all 57 learning outcomes on its exams, and the assessed 
learning outcomes differed from section to section. Only 9 of the 57 learning outcomes were 
examined in all four sections studied. A further 9 learning outcomes were examined in three of 
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the four studied sections, while 14 learning outcomes were examined in two sections, 17 were 
examined in only one section, and 8 learning outcomes were not examined in any of these four 
sections. In this study, we analyzed only exams, and not assignments. It is possible that in some 
sections, some learning outcomes were assessed on assignments but not on exams, which would 
lead us to underrepresent the number of learning outcomes assessed in those sections. 

For each section, we compared the cognitive skill level at which each learning outcome 
was most frequently assessed to the stated cognitive skill level for that particular learning 
outcome. Figure 6 summarizes the difference between the assessed and stated level of the nine 
learning outcomes that were examined in all four sections of the course, organized into ecology, 
genetics, and evolution outcomes. Table 2 summarizes the differences in the cognitive skill level 
at which each learning outcome was most frequently assessed compared to the stated level for 
that particular learning outcome for each section. 

Table 2 

Summary of Differences in the Most Frequen tly Assessed Versus Stated Cogn itive Skill Level of 
Learning outcomes for All Four Sections of Biology 121 ('January-April 2008). 


Section 

Most frequently assessed cognitive 
skill level - at least one level higher 
than stated (%) 

Most frequently assessed cognitive 
skill level - at least one level lower 
than stated (%) 

A 

30 

9 

B 

25 

32 

C 

36 

24 

D 

48 

16 


Even with the considerable variation amongst the sections, several clearly visible trends 
emerged from this analysis. In general, for genetics learning outcomes, the most frequently 
assessed cognitive skill level was similar to the stated cognitive skill level, while for many 
evolution and ecology learning outcomes, the most frequently assessed cognitive skill level was 
noticeably higher than the stated cognitive skill level. For all three subject areas, the 
Comprehension learning outcomes were most likely to be assessed at a higher cognitive skill 
level. This result is not surprising, since over 50% of the course learning outcomes were written 
at this level. These learning outcomes almost always required students to be able to explain or 
describe a concept, while, on exams, these outcomes were often assessed by requiring students to 
apply this concept to a new situation where they had to make a prediction or solve a problem. 
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Figure 6. Differences between the assessed and stated cognitive skill level (i.e., Bloom’s 
Taxonomy Level) of the nine learning outcomes that were examined in all four sections of the 
course, organized into ecology, genetics, and evolution outcomes (Jan-Apr 2008). Negative 
value indicates that the learning objective was assessed at a lower level than stated whereas 
positive value indicates that the learning outcome was assessed at a higher level than stated. A 
value of zero indicates that the learning outcome was assessed at the same level as stated. 

The Evaluation learning outcomes were always assessed at a lower cognitive skill level. Because 
Evaluation questions often require in-depth analysis and complex comparisons amongst related 
topics, it is difficult to construct a question at this level that will fit within the time constraints of 
an exam unless students have had considerable opportunities throughout the term to practice this 
type of question. Therefore, it is not surprising that none of the sections studied had any 
Evaluation exam questions. 

The Effect of Cognitive Skill Level of Exam Questions on Student Performance 

Our preliminary results (from Section B) showed that students tended to perform better 
on exam questions with lower cognitive skill levels. For all 101 students in this section that 
consented to let us use their grades, the average marks on all Comprehension (64%) and 
Application (68%) questions were statistically significantly higher (p < .01 and p < .001 

respectively) than on Analysis questions (58%). Our results suggest that the cognitive skill level 
of a question affected student perfonnance similarly for students in both the top 25% and bottom 
25% of the class (Figure 7). The students in the bottom 25% scored statistically significantly 
lower on Analysis questions (37%) than on Comprehension (48%, p < .01) or Application 
questions (52%, p < .001). Students in the top 25% of the class also scored statistically 
significantly lower on Analysis questions (76%) than on Comprehension (83%, p < .05) or 
Application questions (83%, p < .01). 
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Figure 7. The effect of cognitive skill level (i.e., Bloom’s Taxonomy Level) of exam questions 
on student performance: average marks (%) on Comprehension (II), Application (III), and 
Analysis (IV) final exam questions for students in the top and bottom 25% of Section B (Jan-Apr 
2008) ± standard deviation, n = 25 students in the top 25% and n = 25 students in the bottom 
25%. Asterisks indicate statistically significant differences in students’ marks between the levels 
compared (p < .05). 


To gain further insight, we did a preliminary analysis for two learning outcomes that 
were assessed both at their stated cognitive skill level and at a higher cognitive skill level in 
Section B. One genetics outcome (Genetics 6) was stated as an Application outcome and was 
assessed in two Application final exam questions and in lour Analysis final exam questions. One 
evolution outcome (Evolution 6) was stated as a Comprehension outcome and was assessed in 
one Comprehension question and in one Analysis question. Students in the top 25% of the class 
scored statistically significantly higher (p < .001) on exam questions where Genetics 6 was 
assessed at its stated cognitive skill level (90%) than on exam questions which assessed that 
same outcome at a higher cognitive skill level (79%). They also scored statistically significantly 
higher (p < .05) on the question in which Evolution 6 was assessed at its stated cognitive skill 
level (87%) compared to the question in which the same outcome was assessed at a higher 
cognitive skill level (75%). Although students in the bottom 25% of the class also scored 
statistically significantly higher (p < 0.001) on questions in which Genetics 6 was assessed at its 
stated cognitive skill level (62%) than on questions in which that same outcome was assessed at 
a higher cognitive skill level (36%), there was no significant difference in their marks on the two 
questions assessing Evolution 6 (Figure 8). 
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Learning Outcome 

Figure 8. Student performance on exam questions in which learning outcomes are assessed at the 
stated cognitive skill level (SCSL) versus at a higher cognitive skill level (HCSL) for students in 
the top (n = 25) and bottom (n = 25) 25% of Section B. Asterisks indicate statistically significant 
differences in students’ marks between the levels compared ip < .05). Genetics 6 and Evolution 6 
refer to learning outcomes presented in the Appendix. Error bars indicate standard deviation. 

These differences in student performance could be partly attributed to the general effect 
of the cognitive skill level of a question. However, for students in the top 25% of the class, the 
difference between their average mark on all Application questions and on all Analysis questions 
was only 6.7%, while the difference between their average mark on the questions in which 
learning outcome Genetics 6 was assessed at its stated cognitive skill level and the questions in 
which the same outcome was assessed at a higher level was 11.1%. This discrepancy is even 
more exaggerated for students in the bottom 25% of the class, where the difference between their 
average mark for all Application questions and all Analysis questions (15.3%) is much less than 
the difference in their average mark on the questions in which Genetics 6 was assessed at its 
stated cognitive skill level as opposed to a higher cognitive skill level (26.4%). Our results 
suggest that for all students, assessing a learning outcome at a higher cognitive skill level than 
that suggested by the wording of that learning outcome may have a negative impact on their 
performance above and beyond the general effect of the cognitive skill level of a question. Our 
results reinforce the need to revise the learning outcomes such that they reflect the cognitive skill 
level at which they will be assessed. The reader is cautioned that in this study our sample size 
was small (i.e., only one exam). Thus, generalizing the hypothesis that “the differences in student 
performance could be partly attributed to the general effect of the cognitive skill level of a 
question” requires further analysis with a larger sample size (i.e., more questions on more 
exams). However, this analysis is beyond the scope of the present work. It would, however, be 
interesting to devise a study to test this hypothesis, since Blumberg, Alschuler, and Rezmovic 
(1982) did not find any relationship between student performance and the cognitive skill level of 
multiple choice questions when they examined the same factual content at different cognitive 
skill levels for basic medical sciences material. 
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Recommendations for Learning Outcomes: Wording Modifications to Reflect Intended 
Cognitive Skill Level 

The original learning outcomes were assembled by a committee composed of several 
instructors in Biology 121; these outcomes were revised slightly after each term. After 
identifying all learning outcomes that were assessed at a cognitive skill level different from 
stated, we proposed modifications to the wordings of these learning outcomes to better match the 
cognitive skill level at which they were assessed in the majority of studied sections of the course. 
For most learning outcomes, our recommendations focused entirely on replacing the active verb 
with one that more appropriately reflected how students were assessed for that learning outcome. 
For examples of our recommended wording modifications, see Table 1. In the few cases where 
we identified concepts that were assessed in more than one section but were not mentioned in the 
course learning outcomes, we recommended modifying existing learning outcomes to include 
these concepts or adding an additional learning outcome to address these concepts. 

Figure 5 compares the distribution across each cognitive skill level of the course learning 
outcomes as they were stated (January - April 2008) to that of the recommended learning 
outcomes (Figure 5). With the recommendations, the distribution much more closely resembles 
the distribution of available exam marks across each cognitive skill level for the four studied 
sections of this course (Figure 3). Even though it is unlikely that this course will examine any 
learning outcomes at the Evaluation level, after consultation with the course teaching team, we 
suggested keeping one learning outcome at this level to be assessed as an assignment, where 
sufficient time could be provided to students to address the complexity of an Evaluation 
question. Prior to this study, some sections of the course had open-book exams while the others 
did not. Although this did not appear to affect the distribution of exam marks across each 
cognitive skill level investigated in this study, the teaching team decided to allow students in all 
sections of the course to use one page of summary notes while writing the exams in order to 
provide a more unified learning experience for students. To accommodate this change in course 
policy, we recommended converting all Knowledge level learning outcomes to Comprehension 
level learning outcomes. 

Recommendations for Learning Outcomes: Streamlining 

When composing learning outcomes for a course, it is generally recommended to have no 
more than one learning outcome for each 50-minute class (Jackson, Wisdom, & Shaw, 2003). In 
a standard winter term at UBC, Biology 121 has approximately 36 scheduled 50-minute lecture 
classes. Allowing for the use of three of these class times for midterm exams and review 
sessions, this leaves about 33 classes available for teaching course content. Therefore, we 
suggested reducing the number of course learning outcomes from 57 to approximately 33. We 
made our recommendations for streamlining the course learning outcomes based on the results 
showing which learning outcomes were not examined by any of the four sections studied or were 
examined in only one of the four sections studied. We also recommended merging several 
related learning outcomes which were consistently examined in the same exam question for more 
than one section. For an example of a recommended merging of several related learning 
outcomes, see Table 1. 

We further suggested that the course teaching team agree on approximately 30 core 
learning outcomes which would be taught in all sections of the course. We recommended leaving 
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the remaining 3 learning outcomes open as optional learning outcomes that could be 
personalized by individual instructors to focus on their own specific areas of interest. We 
recommended that the instructors list only the optional outcomes they intend to cover in their 
own section, as opposed to the current practice of providing students with a list of all possible 
optional outcomes. This recommendation would help to prevent students from becoming 
overwhelmed by a long list of learning outcomes. 

Recommendations for Presenting Learning Outcomes to Students 

We suggested that instructors present the relevant learning outcome(s) to students at the 
beginning of each class and when announcing assignments. Before each exam, we recommended 
that instructors provide their students with a list of all the learning outcomes for which they will 
be responsible, to help guide their studying for that exam. This practice was also suggested by 
Simon and Taylor (2009), who found students valued learning outcomes the most in determining 
what they needed to know and were relieved at being given clear direction as to how to focus 
their efforts, both in lectures and in organizing their studying. 

Implications for Teaching and Learning 

One of the most important aspects of this study was to inform instructors of the moderate 
misalignments between learning outcomes and assessments, as well as inconsistencies in 
assessment amongst the different sections, thus enabling the teaching team to make informed 
decisions as they continued their improvement of the course. We kept open communication with 
the teaching team throughout the process and held several meetings from the onset of the study. 
Recently, we presented our results to past and present instructors of the course, discussed the 
implications of our results, and collected feedback from the instructors on how they would use 
this infonnation to promote consistency amongst the different sections of Biology 121. We also 
presented our recommendations for modifications to the course learning outcomes based on our 
comparison of the stated and examined cognitive skill level of each learning outcome. These 
recommendations included modifications to the wording of these learning outcomes such that 
they would more clearly reflect the cognitive skill level at which students would be assessed as 
well as modifications to streamline the learning outcomes. In fact, by rewording and streamlining 
the outcomes and moving to more common exam questions we have been able to increase the 
support for all instructors in the course. This includes a central repository of resources such as an 
instructor’s guide to the learning outcomes, specific examples, and active learning activities that 
can be used in class, and clicker and exam question banks. 

Conclusions 

In this study, we took an evidence-based approach to teaching and learning within the 
scope of a first year biology majors course and demonstrated its feasibility to systematically 
evaluate course learning outcomes and exams using the Blooming Biology Tool (Crowe et al., 
2008). By setting this precedent, we hope to encourage other science educators to consider the 
role of this type of analysis in the process of validating their own course or program curricula. 
We believe that the process we employed here is easily transferable to any course and would be a 
valuable exercise for any course teaching team to undertake. 
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In our study, we were able to identify gaps and overlaps and even slight misalignments 
between the course learning outcomes and exams, and we used this information to make 
recommendations to instructors for revising the learning outcomes to better reflect the intended 
learning for this course. Our preliminary results suggest that student performance was higher 
when the cognitive skill level of assessments matched with that of the stated learning outcomes. 
We therefore suggest to other science educators that employing this type of analysis to better 
align course learning outcomes and exam questions may help to improve student performance. 
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Appendix 

Recommended Biology 121 Learning Outcomes 

ECOLOGY OUTCOMES 

A) Patterns of Biodiversity 

Students should be able to: 

1) Describe the effect of global climate patterns and other physical and biological factors on 
the distribution of species. 

2) Analyze patterns of biodiversity and community structure, given experimental data. 

3) Analyze the impact of human population growth and consumption of resources on 
biodiversity and ecological stability, as well as the ability of conservation plans to offset 
these impacts. 

B) Population/Community Ecology 

Students should be able to: 

4) Estimate the size of a population using different methods, including marked recapture. 

5) Analyze the characteristics of a population, such as age, gender, health, genetic quality, 
as well as the predicted population growth, taking into account the factors that affect 
demographics. 

6) Illustrate population growth using mathematical models, especially the logistic model and 
“boom and bust” cycles. 

7) Determine the life history strategy of different organisms, with respect to fecundity and 
survivorship, given experimental data. 

8) Analyze how biotic and abiotic factors affect population and community structure, and 
the importance of these factors in specific communities. 

9) Analyze changes in community structure that occur as a result of a disturbance (i.e., 
primary and secondary succession). 

C) Ecosystem Ecology 

Students should be able to: 

10) Predict the trophic level and energy source(s) of an organism, as well as the relative 
biomass of groups of organisms and the potential biotic interactions amongst organisms 
in a given ecosystem when provided with information about that ecosystem (for example, 
a food web). 

11) Predict the impact on ecosystems of changes to any or all of the global carbon, nitrogen, 
and water cycles. 

12) Evaluate ways to reduce the ecological footprint of a given individual (make the 
necessary calculations to detennine the effects of modifying each of the factors that 
contribute to the ecological footprint of that individual, then evaluate which of these 
modifications will have the greatest impact). 


http://ir.lib.uwo.ca/cjsotl_rcacea/voll/iss 1/8 
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GENETICS OUTCOMES 

Students should be able to describe how Mendel’s principles of segregation and 

independent assortment are a consequence of chromosome movement in meiosis. 

Core Outcomes: Students should be able to: 

1) Determine whether cells are haploid or diploid. 

2) Illustrate with simple diagrams how cells produce daughter cells during mitosis and how 
diploid cells produce haploid cells during meiosis, including tracing the location of 
alleles during the process. 

3) Demonstrate (using simple diagrams or calculations) how sexual reproduction contributes 
to genetic variation and to degrees of relatedness amongst parents and offspring. 

4) Illustrate with simple diagrams how crossing over results in different gene combinations. 

5) Illustrate how dominant alleles provide sufficient gene function to confer a phenotype 
even when only one copy is present, and how this differs from codominance and 
incomplete dominance. 

6) Calculate expected frequencies in monohybrid, dihybrid, and multihybrid crosses. 

7) Infer the mode of inheritance (e.g., number of genes, dominance, linkage, sex linkage), 
given data from experimental crosses. 

8) Analyze data from a test cross to determine whether genes are linked, as well as the 
recombination frequency of linked genes and the arrangement of these genes on a 
chromosome. 

9) Deduce from a pedigree whether a trait is autosomal or sex linked, dominant or recessive. 

Additional (i.e., noncore) Outcomes: 

al) Integrate all of the above genetics learning outcomes as applicable into our current 
understanding of genetic diversity. 

a2) Describe genetic sex determination in animals and the consequences of having genes on 
the X chromosome. 

a3) Describe the events that occur during the cell cycle, how the cycle is regulated, and how 
errors in regulation can lead to cancer (if cancer is a topic you wish to cover). 

a4) Assess the role of the environment in gene expression (can be linked to cancer if this a 
topic). 
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EVOLUTION OUTCOMES 

Students should be able to describe evolution as a change in allele frequency. 

Students should be able to explain how adaptation occurs by natural selection. 

Students should be able to distinguish between shorter term events: microevolution- 

and longer term events: macroevolution-the pattern of descent. 

Core outcomes: Students should be able to: 

1) Predict how natural selection acting on individuals will affect evolution in populations. 

2) Explain and give examples of how homologies (structural, developmental and molecular) 
provide evidence for evolution. 

3) Predict the relatedness of organisms through interpretation of phylogenetic trees, 
including alternate representations of the same tree. 

The focus of the Hardy-Weinberg equilibrium should not be the equation per se but rather 

the link it provides between genetics, ecology, and evolution. 

4) Calculate the frequency of alleles contributed by a generation in a population, given 
information on the genotype frequencies of that population. 

5) Analyze information on the genotype frequencies of a given population to determine 
whether or not that population is in Hardy-Weinberg equilibrium. 

6) Predict how sources of variation in populations (including different types of selection, 
genetic drift, gene flow, mutation, and inbreeding), will contribute to changing allele 
frequencies in a population. 

7) Explain how evolution is neither directed nor “progressive”, drawing on examples from 
the history of the diversity of life on earth, such as the Cambrian explosion. 

8) Design tests to investigate the evolution of unknown life forms, based on practices 
currently or historically used by biologists to study the history of the diversity of life on 
earth. 

9) Given a specific example or set of data, assess plausible mechanisms of speciation. 

Additional (noncore) outcomes: 

al) Describe the contribution of historical figures (such as Darwin, Wallace, Lamarck, Lyell, 
Malthus, Cuvier, Hutton, Linnaeus, and Mendel) to the theory of evolution. 

a2) Explain with examples why islands and lakes can be evolutionary “hot spots.” 
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